Skip to content

fix(server): turn starts no longer load every handoff's omitted item IDs - #40706

Open
robertexs wants to merge 1 commit into
pingdotgg:mainfrom
robertexs:fix/turn-start-handoff-omitted-ids
Open

robertexs wants to merge 1 commit into
pingdotgg:mainfrom
robertexs:fix/turn-start-handoff-omitted-ids

Conversation

@robertexs

@robertexs robertexs commented Nov 22, 2026 •

Copy link
Copy Markdown

Problem

Starting a turn on a long thread can take about 700 MB of heap, enough to take down a 4 GB server.

Each full_thread_summary handoff stores the ID of every item it omitted in history.omittedItemIds, and delivery copies that list into delivery.omittedItemIds. getTurnStartContext reads every handoff of the thread in full, on every turn start. The lists grow with history length times handoff count: on one install with long-lived threads they are 97% of 950 MB of handoff payloads.

I found it investigating a JavaScript heap out of memory abort on 0.1.8-nightly.20261109.4399. The last turn to be requested was on a thread whose handoffs alone cost 736 MB to load, with about 3.7 GB of heap already in use, and it never finished starting. That link is inferred from timing and size (there was no core dump). The query is the same on main.

Expected: a turn start loads the lists it can use.

Change

getTurnStartContext cuts the two lists in SQLite, before they reach the JS heap, unless this turn can use them:

  • A handoff this turn could still deliver (it targets this run, has no delivery, or goes to this run's provider thread with a pending delivery or a failed, interrupted or cancelled target run) is returned whole, because delivery reads history.omittedItemIds and persists the handoff again.
  • Another handoff to this run's provider thread keeps delivery.omittedItemIds, which coveredItemIds reads. It keeps it for any native thread, so a resume onto an earlier one counts the same.
  • Every other handoff loses both lists. All other fields are untouched.

No schema change and no stored data changes. The windowed read already does the same with json_remove.

Scope

A bug fix for a crash, in one query. It is small and the cause is in the query itself, so I didn't open an issue. #35597 is about the same function, and #35723 lists "still reads every ... context handoff of the thread" under "Not in this change", so this doesn't overlap that PR's fix.

Not changed here: how the lists are written.

Verification

  • Extended the loads startup state without obsolete transcript payloads case in TurnStartReads.test.ts with seven handoff shapes. It fails on main (settled elsewhere: expected [ 'item:omitted' ] to deeply equal undefined) and passes with the fix.
  • Ran this exact SQL beside main's on a real database, read-only, with Node 26.13.1 (70 threads with handoffs, 478 sampled runs, 24,711 rows compared). Every deliverable handoff was identical, every delivery to the run's provider thread was identical, and no other field changed: 0 mismatches. A deliberately broken query produced 151, so the comparison can fail.
  • On the six largest threads (heap held by rows plus parsed payloads, and time to load them):
main this PR
736.3 MB, 1182 ms 27.5 MB, 310 ms
713.4 MB, 920 ms 7.8 MB, 448 ms
180.1 MB, 201 ms 20.4 MB, 104 ms
136.5 MB, 143 ms 12.4 MB, 78 ms
109.3 MB, 113 ms 8.6 MB, 58 ms
109.0 MB, 114 ms 20.9 MB, 48 ms
  • vp run check:changed passes. ProviderTurnStartService, ContextHandoffBudget, ContextHandoffService, ProviderSwitch.integration and Orchestrator.control-reads tests pass (245).
  • Not checked: a live provider turn on a build with this change. Reviewed read-only by GPT-6.2 Astra (no changes requested), which ran the SQL with positional binds on 320 synthetic cases.

Written by Claude Opus 5.5 in Claude Code, running in T3 Code.

🤖 Generated with Claude Code

Each context handoff lists the ID of every item it omitted, and its delivery
copies the list. A turn start read all of them for the thread, so one turn
start on a long thread could take 700 MB of heap.

Cut the lists in SQLite unless this turn can deliver the handoff or needs
its delivery for coverage.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions github-actions Bot added vouch:unvouched Author is not yet trusted in the VOUCHED list. size:S 10-29 changed lines (additions + deletions). labels Nov 22, 2026
@coderabbitai

coderabbitai Bot commented Nov 22, 2026 •

Copy link
Copy Markdown
Contributor

Explain Change →

No actionable comments were generated in the recent review.

Walkthrough

The turn-start context read now trims omitted-item ID lists from context handoffs based on their target and delivery state. Tests cover when those lists remain available and verify that handoff summaries and delivery item IDs are preserved.

Changes: Context handoff turn-start reads

Layer / File(s) Summary
Handoff payload trimming and validation
apps/server/src/orchestration-v2/ProjectionStore.ts, apps/server/src/orchestration-v2/TurnStartReads.test.ts
The SQL read conditionally removes history and delivery omitted-item ID lists based on the handoff target and delivery state. Tests verify which lists remain and check handoff summaries and delivery item IDs.

Estimated code review effort: 2 (Simple) | ~10 minutes

Change: Bug fix

Possibly related issues

  • pingdotgg/t3code#35597: This change trims unused omission-ID lists in getTurnStartContext, one of the reads described in the issue. It does not change the other reads or index choice mentioned there.

Suggested reviewers: yash-singh1, juliusmarminge

Merge Risk: Minimal · up to 40084

Turn-start reads now trim omitted-item ID lists in SQL, so long threads no longer load every handoff's lists into memory. The handoffs that delivery or coverage needs keep their full lists. Tests cover those retention rules, and no concrete defect was found, so merge risk is minimal. A live provider turn was not exercised.

Pre-merge checks: Passed 4

Check name Status Explanation
Title check Passed The title names the main change: turn-start reads no longer load every handoff's omitted item IDs. The raw summary says getTurnStartContext trims omittedItemIds in SQL before decoding, which matches t…
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Description check Passed The description follows the repository template closely. It has all four required sections (Problem, Change, Scope, Verification) and fills each one with specific content. The Problem section gives th…
Full details: Title check

Explanation

The title names the main change: turn-start reads no longer load every handoff's omitted item IDs. The raw summary says getTurnStartContext trims omittedItemIds in SQL before decoding, which matches the title. The title follows the conventional commit format and is specific enough for a teammate scanning history.

Full details: Description check

Explanation

The description follows the repository template closely. It has all four required sections (Problem, Change, Scope, Verification) and fills each one with specific content. The Problem section gives the symptom, the affected query, and reproduction context (a heap-out-of-memory abort on a named nightly build, with the link to the crash marked as inferred). The Change section explains the SQLite trimming rules for each handoff category and states that no schema or stored data changes. The Scope section explains why no issue was opened and cross-references related PRs. The Verification section lists the extended test case, a read-only comparison against a real database with 0 mismatches, before/after heap and timing measurements, the test suites run, and what was not checked (a live provider turn). The author also notes the title and the agent/model used, which the template asks for. Minor gaps: the title is not restated in the body, and the template asks to link a bug issue, which the author explicitly declined to do with a justification. Those gaps are non-critical.

Finishing Touches

Generate unit tests (beta)

  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts
Recent review info

Run configuration

Commits

Reviewing files that changed from the base of the PR and between b9d0f70 and 4008474.

Files selected for processing: 2

  • apps/server/src/orchestration-v2/ProjectionStore.ts
  • apps/server/src/orchestration-v2/TurnStartReads.test.ts

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 9 remain after this review.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:S 10-29 changed lines (additions + deletions). vouch:unvouched Author is not yet trusted in the VOUCHED list.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant